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IN THE CLAIMS: 

The text of all pending claims, (including withdrawn claims) is set forth below. Cancelled 
and not entered claims are indicated with claim number and status only. The claims as listed 
below show added text with underlining and deleted text with otr i kothrough . The status of each 
claim is indicated with one of (original), (currently amended), (cancelled), (withdrawn), (new), 
(previously presented), or (not entered). 

Please CANCEL claim 33 in accordance with the following: 

1 . (PREVIOUSLY PRESENTED) A robot system having an image processing 
function for determining orientation, or orientation and position of a robot operation on one of a 
plurality of objects, the system comprising: 

a robot; 

a first image capturing device capturing image data of the plurality of objects containing 
respective images of the objects; 

a memory storing reference models, each comprising an image of a reference object 
captured by said image capturing device in a different direction, and for each reference model 
storing information of the capturing direction of its associated image and information of an 
orientation of the robot with respect to the reference object, the information of the capturing 
direction representing a rotational posture of the reference object relative to the robot, said 
reference object being one of the plurality of objects or an object having a shape identical to that 
of one of the plurality of objects; and 

a processor to perform matching on the image data containing images of the plurality of 
objects captured by said first image capturing device with each of said reference models 
successively to select one object having an image matched with one of said reference models, 
and to determine orientation, or orientation and position of the robot operation based on the 
image of the selected one object, based on said one reference model and the information of its 
associated capturing direction, and based on the information of the orientation of the robot 
operation with respect to the reference object that is associated with said one reference model. 



2 



Serial No. 09/546,392 

2. (PREVIOUSLY PRESENTED) A robot system having an image processing 
function according to claim 1, wherein said reference models are obtained from a part of the 
image data of the reference object. 

3. (ORIGINAL) A robot system having an image processing function according to 
claim 1, wherein said reference models are obtained by processing the image data of the 
reference object. 

4. (ORIGINAL) A robot system having an image processing function according to 
claim 1, wherein said first image capturing device comprises a camera for capturing two- 
dimensional image data. 

5. (ORIGINAL) A robot system having an image processing function according to 
claim 4, wherein said image data of the reference object are captured by said camera from a 
predetermined distance. 

6. (PREVIOUSLY PRESENTED) A robot system having an image processing 
function according to claim 1, further comprising: 

a second image capturing device; wherein 

said robot situates said second image data capturing device to have said determined 
orientation or to have said determined orientation and said determined position with respect to 
the selected one object, and wherein 

said processor processes second image data captured by said second image capturing 
device to detect position and/or rotational posture of the selected one object with respect to said 
second image data capturing device. 

7. (PREVIOUSLY PRESENTED) A robot system having an image processing 
function according to claim 1 further comprising: 

a second image capturing device for obtaining three-dimensional position; wherein 
said robot situates said second image data capturing device to have said determined 
orientation or to have said determined orientation and said determined position with respect to 
the selected one object, so that said second image data capturing device is directed to a 
characterizing portion of the object; and wherein 

said processor detects three-dimensional position and/or posture of the selected one 
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object based on three-dimensional position of said characterizing portion obtained by said 
second image capturing device. 

8. (PREVIOUSLY PRESENTED) A robot system having an image processing 
function according to claim 6, wherein said first image data capturing device is used as said 
second image data capturing device. 

9. (PREVIOUSLY PRESENTED) A robot system having an image processing 
function according to claim 6, wherein said second image capturing device comprises a three- 
dimensional visual sensor of spot-light scanning type capable of measuring distance between 
the sensor and an object. 

10. (PREVIOUSLY PRESENTED) A robot system having an image processing 
function according to claim 6, wherein said second image data capturing device comprises a 
structured-light unit for irradiating a structured light on the selected object and capturing an 
image of the object including the irradiated light on the object 

1 1 . (PREVIOUSLY PRESENTED) A robot system having an image processing 
function according to claim 7, wherein said robot operation is an operation of picking up the 
selected one object from the plurality of objects, some of which are overlapped with each other. 

1 2. (PREVIOUSLY PRESENTED) A robot system having an image processing 
function for determining orientation, or orientation and position of a robot operation on one of a 
plurality of objects of plural kinds, where the orientation of the operation corresponds to a 
determined orientation of the one object, where the determined orientation is a rotational posture 
of the one object, the system comprising: 

a robot; 

a first image capturing device capturing image data of the plurality of objects containing 
respective images of the objects; 

a memory storing reference models, each comprising images of each of different kinds of 
reference objects corresponding to images captured by said first image capturing device, and 
storing indicia of the kinds respectively associated with said reference models, and information 
of a different orientation of the robot with respect to each of the different images of the reference 
object of each kind, where the captured information of orientation comprises information of a 
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rotational posture of the reference object relative to the robot, each of said reference objects 
being one of the kinds of the plurality of objects or having a shape identical thereto; and 

a processor to perform matching on the image data containing images of the plurality of 
objects captured by said first image capturing device with each of said reference models 
successfully to select one object having an image matched with one of said kinds of the 
reference models, and to determine orientation, or orientation and position of the robot 
operation, the determining based on the image of the selected one object, based on said one 
reference model, based on the indicia of the kind associated with said one reference model and 
the information of the orientation of the robot operation with respect to the reference object 
associated with said one reference model of said one kind. 

1 3. (PREVIOUSLY PRESENTED) A robot system having an image processing 
function according to claim 12, wherein said reference models are obtained from a part of the 
image data of the reference object. 

14. (ORIGINAL) A robot system having an image processing function according to 
claim 12, wherein said reference models are obtained by processing the image data of the 
reference object. 

15. (ORIGINAL) A robot system having an image processing function according to 
claim 12, wherein said first image capturing device comprises a camera for capturing two- 
dimensional image data. 

16. (ORIGINAL) A robot system having an image processing function according to 
claim 15, wherein said image data of the reference object are captured by said camera from a 
predetermined distance. 

1 7. (PREVIOUSLY PRESENTED) A robot system having an image processing 
function according to claim 12, further comprising a second image capturing device, wherein 

said robot situates said second image data capturing device to have said determined 
orientation or to have said determined orientation and said determined position with respect to 
the object, and wherein 

said processor processes second image data captured by said second image capturing 
device to detect position and/or posture of the selected one object with respect to said second 
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image data capturing device. 

1 8. (PREVIOUSLY PRESENTED) A robot system having an image processing 
function according to claim 12, further comprising: 

a second image capturing device for obtaining three-dimensional position; wherein 
said robot situates said second image data capturing device to have said determined 

orientation or to have said determined orientation and said determined position with respect to 

the selected one object, so that said second image data capturing device is directed to a 

characterizing portion of the object; and wherein 

said processor detects three-dimensional position and/or posture of the selected one 

object based on three-dimensional position of said characterizing portion obtained by said 

second image capturing device. 

1 9. (PREVIOUSLY PRESENTED) A robot system having an image processing 
function according to claim 17, wherein said first image data capturing device is used as said 
second image data capturing device. 

20. (PREVIOUSLY PRESENTED) A robot system having an image processing 
function according to claim 17, wherein said second image capturing device comprises a three- 
dimensional visual sensor of spot-light scanning type capable of measuring distance between 
the sensor and an object. 

21 . (PREVIOUSLY PRESENTED) A robot system having an image processing 
function according to claim 17, wherein said second image data capturing device comprises a 
structured-light unit for irradiating a structured light on the selected one object and capturing an 
image of the object including the irradiated light on the object. 

22. (PREVIOUSLY PRESENTED) A robot system having an image processing 
function according to claim 18, wherein said robot operation is an operation of picking up the 
selected one object from the plurality of objects, some of which are overiapped with each other. 

23. (PREVIOUSLY PRESENTED) A method for automatically determining an 
arrangement of a workpiece relative to a robot, where the determined arrangement comprises at 
least rotational posture arrangement of the workpiece relative to the robot, the method 
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comprising: 

storing reference images corresponding to images of the workpiece or an object so 
shaped (workpiece/object) and reference arrangement information indicating arrangements of 
the robot and workpiece/object relative to each other when the images were captured, the 
reference arrangements comprising rotational arrangements of the workpiece relative to the 
robot; 

from a known arrangement of the robot, capturing a working image of the workpiece 
among a plurality of randomly arranged workpieces with an imaging device; 

finding one of the reference images that matches the workpiece in the working image; 

and 

determining an arrangement of the robot relative to the workpiece based on information 
indicating the known arrangement of the robot, and based on the reference arrangement 
information corresponding to the found reference image, where the determined arrangement 
comprises rotational arrangement of the workpiece relative to the robot. 

24. (PREVIOUSLY PRESENTED) A method according to claim 23, wherein reference 
images and reference arrangement information is obtained for workpieces/objects of different 
shapes, and wherein the finding comprises first determining that a reference image of one of the 
different shapes matches the working image of the workpiece. and then finding one reference 
image of the shape that best matches the working image. 

25. (PREVIOUSLY PRESENTED) A method according to claim 23, wherein the robot 
is used to capture the reference images, and wherein the reference arrangement information 
represents arrangements of the robot when capturing the reference images. 

26. (PREVIOUSLY PRESENTED) A method according to claim 23, wherein a second 
imaging device is affixed to the robot and is used to determine additional arrangement 
information used to determine the known arrangement of the robot relative to the workpiece. 

27. (PREVIOUSLY PRESENTED) A robot system having an image processing 
function according to claim 7, wherein said first image data capturing device is used as said 
second image data capturing device. 

28. (PREVIOUSLY PRESENTED) A robot system having an image processing 
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function according to claim 7, wherein said second image capturing device comprises a three- 
dimensional visual sensor of spot-light scanning type capable of measuring distance between 
the sensor and an object. 

29. (PREVIOUSLY PRESENTED) A robot system having an image processing 
function according to claim 7, wherein said second image data capturing device comprises a 
structured-light unit for irradiating a structured light on an object and capturing an image of the 
object including the irradiated light on the object. 

30. (PREVIOUSLY PRESENTED) A robot system having an image processing 
function according to claim 18, wherein said first image data capturing device is used as said 
second image data capturing device. 

31 . (PREVIOUSLY PRESENTED) A robot system having an image processing 
function according to claim 18, wherein said second image capturing device comprises a three- 
dimensional visual sensor of spot-light scanning type capable of measuring distance between 
the sensor and an object. 

32. (PREVIOUSLY PRESENTED) A robot system having an image processing 
function according to claim 18, wherein said second image data capturing device comprises a 
structured-light unit for irradiating a structured light on an object and capturing an image of the 
object including the irradiated light on the object. 



33. (CANCELLED) 
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